Papers by Lucy Lu Wang
S2ORC: The Semantic Scholar Open Research Corpus (2020.acl-main)
Copied to clipboard
| Challenge: | Academic papers are an increasingly important textual domain for natural language processing (NLP) research. |
| Approach: | They propose to aggregate 81.1M English-language academic papers into a unified source . they hope this resource will facilitate research and development of tools for text mining over academic text. |
| Outcome: | The proposed corpus includes metadata, abstracts, bibliographic references, and structured full text for 8.1M open access papers. |
Generating Scientific Claims for Zero-Shot Scientific Fact Checking (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for scientific fact checking require domain expertise and time consuming. |
| Approach: | They propose a new supervised method for generating claims from scientific sentences and a novel method for negating claims. |
| Outcome: | The proposed method improves on existing methods on biomedical claims and negations. |
Construction of the Literature Graph in Semantic Scholar (N18-3)
Copied to clipboard
Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, Oren Etzioni
| Challenge: | Fig. 1 summarizes a scalable system for organizing published scientific literature into a heterogeneous graph . authors describe methods used to enable semantic features in www.semanticscholar.org . |
| Approach: | They describe a scalable system for organizing published scientific literature into a heterogeneous graph to facilitate algorithmic manipulation and discovery. |
| Outcome: | The proposed system can be deployed on a scalable platform and report empirical results for each task. |
VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groups (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent work has improved extraction accuracy by incorporating elementary layout information, for example, each token’s 2D position on the page, into language model pretraining. |
| Approach: | They propose a method that explicitly models VIsual LAyout (VILA) groups, that is, text lines or text blocks, to further improve extraction accuracy. |
| Outcome: | The proposed methods show that inserting special tokens denoting layout group boundaries can lead to a 1.9% Macro F1 improvement in token classification. |
MedICaT: A Dataset of Medical Images, Captions, and Textual References (2020.findings-emnlp)
Copied to clipboard
Sanjay Subramanian, Lucy Lu Wang, Ben Bogin, Sachin Mehta, Madeleine van Zuylen, Sravanthi Parasa, Sameer Singh, Matt Gardner, Hannaneh Hajishirzi
| Challenge: | Existing largescale datasets explicitly exclude compound figures . existing systems lack this ability to identify relevant subfigures . |
| Approach: | They propose a dataset of medical images in context that allows figure-to-text alignment . they use captions, inline references and manually annotated subfigures for compound figures . |
| Outcome: | The proposed dataset demonstrates the utility of inline references in image-text matching. |
MSˆ2: Multi-Document Summarization of Medical Studies (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets for multi-document summarization (MDS) are either in the general domain, such as WikiSum, or very small such as DUC 1 or TAC 2011 . Existing systems for summarizing biomedical literature take 1-2 years to complete . |
| Approach: | They propose to use a multi-document summarization system based on BART to assess the quality of the summarized biomedical literature. |
| Outcome: | The proposed system has high summarization quality, but significant work remains to achieve it. |
Illusions of the Gold Standard: A Large-scale Analysis of Human Evaluation Protocols for Long-form Text Generation (2026.acl-long)
Copied to clipboard
Katelyn X. Mei, Yi-Li Hsu, Minjoon Choi, Zongwan Cao, Chenjun Xu, Bingbing Wen, Su Lin Blodgett, Lucy Lu Wang
| Challenge: | a large-scale analysis of human evaluation protocols for long-form generation tasks is lacking in current practice . current protocols lack proper standardization and operationalization, which can limit validity of evaluation . |
| Approach: | They conduct a large-scale analysis of human evaluation protocols for long-form generation tasks in *CL conference papers from 2023–2025. |
| Outcome: | The proposed evaluation protocols lack standardization and operationalization, the authors show . they also find that the evaluation protocols are inadequate for specific domains and tasks . |
Literature-Augmented Clinical Outcome Prediction (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to clinical outcome prediction use only clinical notes and general biomedical literature. |
| Approach: | They propose to retrieve patient-specific medical literature and incorporate it into predictive models by combining clinical notes with language models. |
| Outcome: | The proposed approach boosts predictive performance on three important clinical tasks in comparison to strong LM baselines, increasing F1 by up to 5 points and precision@Top-K by a large margin of over 25%. |
Fact or Fiction: Verifying Scientific Claims (2020.emnlp-main)
Copied to clipboard
David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi
| Challenge: | SciFact is a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. |
| Approach: | They construct a dataset of 1.4K scientific claims paired with evidence-containing abstracts annotated with labels and rationales to test their system. |
| Outcome: | The proposed system can verify claims related to COVID-19 by identifying evidence from the CORD-19 corpus. |
ROBOTO2: An Interactive System and Dataset for LLM-assisted Clinical Trial Risk of Bias Assessment (2025.emnlp-demos)
Copied to clipboard
Anthony Hevia, Sanjana Chintalapati, Veronica Ka Wai Lai, Nguyen Thanh Tam, Wai-Tat Wong, Terry P Klassen, Lucy Lu Wang
| Challenge: | Clinical trials provide the highest quality evidence for clinical care . applying ROB2 is time-consuming, taking trained reviewers 30+ minutes per clinical trial report. |
| Approach: | They propose an open-source platform for large language model-assisted risk of bias assessment of clinical trials. |
| Outcome: | The proposed platform enables rapid and reproducible clinical trial annotations. |
Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Psychology research has shown that humans are poor at estimating their performance on tasks, tending towards underconfidence on easy tasks and overconfidence on difficult tasks. |
| Approach: | They propose to use a self-assessment method to assess confidence in large language models (LLMs) they propose to ask for the answer separately and then use them to improve their accuracy. |
| Outcome: | The proposed method improves confidence calibration and interpretability in QA tasks with different personas. |
SUPP.AI: finding evidence for supplement-drug interactions (2020.acl-demos)
Copied to clipboard
Lucy Lu Wang, Oyvind Tafjord, Arman Cohan, Sarthak Jain, Sam Skjonsberg, Carissa Schoenick, Nick Botner, Waleed Ammar
| Challenge: | Dietary supplements are used by a large portion of the population, but information on their pharmacologic interactions is incomplete. |
| Approach: | They propose an application to search evidence sentences extracted from the literature to identify supplement-drug interactions. |
| Outcome: | The proposed model extracts supplement information and identifies interactions using labeled DDI data. |
SciFact-Open: Towards open-domain scientific claim verification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current scientific claim verification systems can achieve very strong performance on limited contexts, in some cases approaching human agreement. |
| Approach: | They propose to pool and annotate top predictions from four state-of-the-art scientific claim verification models to evaluate their performance against large corpora. |
| Outcome: | The proposed system performs well on a corpus of 500K scientific abstracts. |
Automated Metrics for Medical Multi-Document Summarization Disagree with Human Evaluations (2023.acl-long)
Copied to clipboard
Lucy Lu Wang, Yulia Otmakhova, Jay DeYoung, Thinh Hung Truong, Bailey Kuehl, Erin Bransom, Byron Wallace
| Challenge: | Prior work has shown that models may exploit shortcuts that are difficult to detect using standard n-gram similarity metrics such as ROUGE. |
| Approach: | They propose to use human-assessed summary quality facets and pairwise preferences to improve MDS evaluation methods. |
| Outcome: | The proposed methods improve the quality of literature review summarization models . they use human-assessed summary quality facets and pairwise preferences . |
MultiVerS: Improving scientific claim verification with weak supervision and full-document context (2022.findings-naacl)
Copied to clipboard
| Challenge: | a new approach to scientific claim verification uses a document-level fact-checking label to label scientific documents . a multitask approach combines a shared encoding of the claim and document context . |
| Approach: | They propose a system which predicts a fact-checking label and identifies rationales in a multitask fashion based on a shared encoding of the claim and full document context. |
| Outcome: | The proposed approach outperforms baselines on three scientific claim verification datasets . it can learn from instances annotated with a document-level fact-checking label, but lacking sentence-level rationales based on the datasets. |